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Abstract 

Goal, Scope and Background. Many disciplines, amongst them 
LCIA, environmental impact and external cost assessments, are 
often faced with evaluating trade-offs between two or more al¬ 
ternative options in terms of a range of incommensurable indi¬ 
cators. Using process modeling and valuation, these indicators 
are quantified at mid- or endpoint levels. Recent discussion 
amongst LCA experts showed that because of the mutually exclu¬ 
sive aspects of uncertainty and relevance, the midpoint/endpoint 
debate is controversial and difficult to reconcile. This article is 
aimed at a more quantitative analysis of mid- and endpoint im¬ 
pacts, and the implications of uncertainty for decision-making. 

Methods. The consequences for decision-making of uncertain¬ 
ties of endpoints are analysed quantitatively for the example of 
ExternE results, by employing statistical hypothesis testing. The 
Analytic Hierarchy Process (AHP) is then used to demonstrate 
the use of multi-criteria techniques at midpoint levels. 

Results and Discussion. Statistical hypothesis testing at the end¬ 
point level shows that for the ExternE example, probabilities of 
mistakenly favouring one alternative over another when they 
are in reality indistinguishable can be as high as 80%. There¬ 
fore, the best estimate of external cost is inadequate for most 
policy making purposes. Indicators at midpoint levels are more 
certain, but since they are only 'proxy attributes', they carry a 
hidden uncertainty in their relevance. 

Conclusion. If endpoint information is too uncertain to allow a 
decision to be made with reasonable confidence, then the as¬ 
sessment can be carried out in midpoint terms. However, mid¬ 
point indicators are generally further removed from people's 
experience, and less relevant to the question that people actu¬ 
ally want to solve. Nevertheless, if this ultimate question is un¬ 
answerable (within the certainty required by the decision-maker), 
a decision can be made on the basis of stakeholders' subjective 
judgments about the more certain midpoint levels. The crucial 
point is that these judgments are able to intuitively incorporate 
many aspects that impact modeling and valuation has trouble 
quantifying, such as perceived risk, distribution of burdens and 
benefits, equity, ethical, moral, religious and political beliefs and 
principles, immediacy and reversibility of potential impacts, 
voluntariness, controllability and familiarity of exposure, or 
perceived incompleteness of human knowledge. 

Keywords: Endpoint; externality assessment; midpoint; multi¬ 
criteria decision-making; uncertainty; valuation 


Introduction 

There is an abundance of situations, where a comparative 
assessment of two or more alternative options (products, 
companies, projects, industry sectors), in terms of a range 
of social, economic and environmental indicators has to be 
carried out. Examples are external cost assessment, indus¬ 
try benchmarking, environmental impact assessment, or life- 
cycle assessment (LCA). 

Indicators featuring in these assessments are often not read¬ 
ily quantifiable, such as biodiversity or community partici¬ 
pation. Even if they are quantifiable, many indicators such 
as noise and emissions are incommensurable, that is they 
possess different units that cannot be added together. For 
example, a life-cycle impact assessment can conclude with 
several numbers pertaining to different impact categories. 

In a comparative assessment based on more than one cat¬ 
egory, one alternative will generally not score better than 
another in all categories, but instead there are likely trade¬ 
offs between the alternatives, so that a clear preference can¬ 
not readily be identified. While according to ISO 14042 the 
aggregation of different impacts into a single number is only 
optional, a single, aggregate measure of impact or perform¬ 
ance, which is easy to interpret and communicate to stake¬ 
holders and the general public, is often desired by decision¬ 
makers. As a result, a number of concepts have been devised 
that yield such measures, for example the sustainable proc¬ 
ess index [1], the ecological footprint [2], or the eco-indica- 
tor [3]. 1 Approaches taken in some of these concepts are 
often roughly classified into two categories: process modeling 
and valuation. 

Process modeling is guided by a systemic-holistic view: an 
empirical causal description of the interactions of all rel¬ 
evant observable natural and socio-economic processes is 
attempted. In LCA for example (see Fig. 4 in [9]), 'interven¬ 
tions' (such as CO, and other emissions, land, energy and 
resource use etc) are defined, and followed through com¬ 
plex 'impact pathways' using fate modeling, via 'midpoints' 
(sea level rise, ecotoxicology, eutrophication etc) to 'end- 


1 Such indicators have especially been discussed within LCA. For a sum¬ 
mary see [4], for a comparative assessment [5-7], for a critique [8], 
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Fig. 1 : Example for two mutually exclusive impact pathways ending in a 
common endpoint measure. Conversions between outputs (energy) and 
impact measures (emissions, radioactivity etc) are represented by char¬ 
acterisation factors f t The uncertainty of these factors causes the uncer¬ 
tainty of subsequent impact measures to increase. Eventually, the high 
degree of overlap of the bell-shaped value distributions precludes deci¬ 
sion-making 

points' 2 (damage and loss of human and non-human life, 
loss of resources etc) relating to certain safeguard objects 
(human health, ecosystem health, resource base). 3 Impacts 
can be determined as ex-post average values, or as ex-ante, 
marginal effects of changes in underlying parameters. 

An impact pathway of a nuclear power plant could for ex¬ 
ample start at the release of radioactivity during normal 
operation and as a consequence of accidents, via exposure 
to human and non-human life, leading to doses in tissue, 
causing incidents of cancer or other radiation-related dis¬ 
eases, resulting in lost or injured lives (Fig. 1). A similar 
pathway could be set up for a fossil-fuelled power plant, 
with radioactivity stemming from the fly ash. A completely 
separate, or mutually exclusive impact pathway for the fos¬ 
sil-fuelled plant could start at the emission of greenhouse 
gases, raising concentrations and radiative forcing in the 
atmosphere, leading to global warming, climate change, veg¬ 
etation zone shifts, and sea level rise, causing population 
displacement, harvest failures, and water quality and avail¬ 
ability deterioration, resulting in lost or injured lives. Once 
again, a similar pathway could be set up for the nuclear 
plant, with greenhouse gases embodied in the plant's com- 


2 In this work, the term endpoint' is applied to aggregate measures at the 
end of one, or several converging impact pathways. This usage may be 
different from other usages of this term, for example in Environmental 
Risk Assessment. It also differs from one reviewer's view that endpoints 
are characterised by the fact that they must be perceivable for decision¬ 
makers. This reviewer also remarked that the ecoindicator [3] and the 
ExternE study [15,16] do not necessarily fulfil this condition. The ques¬ 
tion asked in this work is whether prohibitive uncertainty precludes mov¬ 
ing further along the impact pathways, no matter whether endpoints are 
perceivable or not. 

3 An alternative terminology could be 'stresses' affecting 'environmental 
quality' leading to 'receptor effects' that can be converted in to monetary 
damages [10]. 


ponents, or emitted during the plant's operation. While the 
radioactivity impacts represent the dominant pathway for 
the nuclear plant, greenhouse gas impacts constitute the pri¬ 
ority pathway for the fossil plant. 

The term 'valuation' as used in this work shall refer to nor¬ 
mative techniques attempting to elicit human preferences, 
judgment and values, in relation to commodities or (dis)- 
amenities not traded in a market. The perspective taken in 
valuation methods is essentially human-centric, and thus 
deals with revealed preferences (hedonic price, travel or avert¬ 
ing cost methods) or stated preferences (contingent valua¬ 
tion or conjoint analysis). A typical valuation exercise could 
aim at establishing either a relative ranking of attributes and 
alternatives that is consistent with stakeholders' behaviour, 
utility, or preferences, or it could attempt monetisation by 
determining for example willingness to pay, willingness to 
accept (compensation) 4 , shadow prices, or net present value 
in absolute (monetary) terms. 

A mixture of process modeling and valuation is often ap¬ 
plied, with the latter techniques usually being applied to 
aggregate endpoint levels [3,13,14]. An example for such 
an approach is the well-known ExternE project of the Euro¬ 
pean Commission and the US Department of Energy [15,16], 
which seeks to develop an impact path methodology for the 
estimation and comparison of the environmental external¬ 
ity cost of various fuel cycles for electricity generation. While 
impact pathways are followed through to lost and injured 
lives, the conversion into associated external cost (the 
endpoint) is based on valuation. Hertwich [17] notes that 
values are in principle present even in parts of process 
modeling, an example being Global Warming Potentials, 
which are based on a chosen time horizon. 

This article is aimed at analysing the implications for deci¬ 
sion-making of uncertainty at mid- and endpoint stages in 
impact or externality assessments. The following Section sets 
the scene by describing the causes of such uncertainties. The 
consequences for decision-making of uncertainties at end¬ 
points are then quantified for the example of ExternE re¬ 
sults, by employing statistical hypothesis testing. The Ana¬ 
lytic Hierarchy Process (AHP) is then used to demonstrate 
the use of multi-criteria techniques at midpoint levels. There¬ 
after, advantages and disadvantages of endpoints and mid¬ 
points are discussed as certainty/relevance trade-offs, rec¬ 
ommendations are made, and the article is concluded. A 
general aim of the article is to be a source of links to the 
literature, through frequent references for further reading 
on the numerous aspects of the topic. 

1 Uncertainty in Impact and Externality Assessments 

Determining impacts, externalities or damage cost using 
process modeling and valuation is associated with consider¬ 
able uncertainties, 5 arising from a range of (overlapping) 


4 Willingness to pay and willingness to accept can differ due to income 
and substitution effects [11], and due to loss aversion and status quo 
bias [12]. 

5 Similar qualifications are also applicable to attempts to valuate biodivers¬ 
ity, natural capital, or ecosystem services [18,19], 
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Table 1: Causes and examples of uncertainty within process modeling and valuation. Acronyms in brackets denote the type of uncertainty [20,21]: 
C - choice/subjectivity, E - epistemological/unknown facts, M - methodological/model, P - technical/parameter 

Cause of uncertainty - Examples 
PROCESS MODELING 

Complexity of nature, lack of knowledge, and inadequacy of models [93] 

Linearisation of non-linear relationships between processes [9] and complicated temporal effects [94], such as 

— the global carbon cycle and atmospheric dynamics [24], especially for chemically active gas species [36,41] (M) 

— disturbance of ecosystems and resilience [95] (M) 

— extrapolation of dose-response relationships towards low doses, especially integration of stochastic effects of low-level radiological doses 
[96] (M) 

Omission of interactions between impact compartments when using partial models (for example interdependencies between ecological 
communities and habitats can facilitate complex ripple effects flowing on from an initial disturbance of one ecological component) (E) 

Unknown impact pathways (E) 

Prediction of future effects 

Uncertain future temperature and sea levels, and vegetation impacts due to climate change (P) 

Uncertain future doses from long-lived radionuclides, to uncertain future receptor populations (P) 

Episodic events 

Uncertain probability of a major reactor accident, which is due to a scarcity of data on previous incidents or component failure (P) 

Uncertain receptor or environment conditions at time of episodic events (P) 

Variability 

Uncertain exposure to radioactivity from a major reactor accident under variable prevailing winds and non-uniform receptor densities (P) 

Top-down approaches ignoring site specificity [97,98] a (M) 

Limitation in depth and/or breadth, and spatial and temporal boundaries 

Depth: the systematic error in all life-cycle inventories compiled using process analysis, which is caused by the truncation of the system by a 
finite boundary that usually includes upstream processes of only first or second order [43,99,100] (M) 

Breadth: the omission of (M) 

— processes leading to damage 

— system components that during their production or operation cause impacts 

— indicators or components thereof, e.g. the omission of ecosystem impacts ([48] p. 66), non-environmental impacts such as military spending 
for energy security [36], or cost of nuclear proliferation [57] 

setting of spatial and temporal boundaries that are arbitrary 11 , or inconsistent across methods and/or alternatives; 

Double-counting 

Unknown synergistic or cumulative effects of different pollutants, mix of technologies, or different indicators (E) 

Overlap of use values with non-use values, if the latter are associated with unknown (because indirect or otherwise hidden) service flows from 
ecosystems to people (M, C) 

VALUATION 

Sensitivity of results to choices of discount rates in monetisation 0 (C) 

Biases and inconsistencies occurring during elicitation of indicator weights: people may construct preferences while being interviewed, or may 
state preferences that contradict utility theory 01 (C) 

Variability because opinions vary between people, and also change over time: results of valuation exercises can rarely be generalised, and are 
prone to criticism and dispute [101] (P) 

Omission of non-use values, such as existence of pristine nature, bequests to future generations, or optional uses 0 (M) 

Stochastic uncertainty resulting out of imperfect preference measurements * 1 (P) 

0 Using fate and exposure modeling, Hertwich et al. [34] show that varying landscape characteristics causes less than 10% of total variability of potential 
doses from toxic emissions, but that the behaviour of the chemical in its release compartment, and exposure routes are the main uncertainty sources. 
Eyre [83] points out that site independence may be valid for high-stack emissions, but not necessarily for emissions from transport. 
b For example 100,000 years for long-lived radionuclides ([43] p. 2), or 100-year time horizon for greenhouse gases [102], 

0 Varying the discount rate has substantial effects on damage cost. For the extreme example of impacts from radionuclides with half lives exceeding 
100,000 years, choices are so sensitive that even very small discount rates lead to far-future damages being negligible. 
d See the interesting literature on violations of classical utility theory by phenomena such as fanning-out, preference reversal and framing effects 
[103-105], extremeness aversion and contrast effects [106], as well as attempts to reconcile economic theory with evidence from psychology 
through ever-adapting new utility formulations [107,108]. 

0 This argument can refer to diverging point of views [109,110], ie either that non-use values have not been considered adequately and should be fully 
evaluated using for example Contingent Valuation [111], that they cannot be reliably measured and therefore should not be considered [101], or that 
existence values should not be conflated with monetisation because they have nothing to do with benefits or welfare but with beliefs and rights [76,77], 

1 The European Commission ([48] p. 87) argues that “there is no sensible way of attaching probabilities to judgments, scenarios of the future, the 

'correctness' of ethical choices or the chances of error", and that "there is no reason to expect that a statistical distribution has any meaning when 
attempting to take into account the possible variability in these parameters". On the other hand, if one accepts that 'willingness to pay' for example is 
amenable to measurement, preferences elicited from a respondent sample may range around a central value. For example, if the mean of 'morbidity 
cost' or 'utility loss from pain and suffering' across a whole population were the quantity sought in an externality study, statistical uncertainties may arise 
out of the respondent sample not representing this population. Another example for stochastic uncertainty within valuation is the value of a statistical life 
(VSL) or years-of-life-lost (YOLL), which may or not depend on a statistically uncertain age at death, or on health status [112,113]. 


causes (Table l ). 6 Some of these causes of uncertainty may be 
stochastic, that is parametric as a statistical distribution around 

6 Flofstetter [20] classifies these into 'epistemological uncertainty' (sys¬ 
tematic lack of knowledge about nature and the future), 'subjectivity' (value 
judgments), and 'technical and methodological uncertainty 1 (statistical 
variation of parameters). Huijbregts [21] distinguishes parameter, model, 
and choice uncertainty, and also treats unknown spatial and temporal 
variability separately from uncertainty. 


a mean value. Other causes, for example in inadequate mod¬ 
els or predictions of the future, may be systematic, that is yield¬ 
ing too high or too low values (unknown impact pathways 
being an example of the latter). Naturally, stochastic proc¬ 
esses lend themselves to statistical analysis, and for the sake of 
simplicity, these will be the focus in this work. Note that sys¬ 
tematic uncertainties due to unknown impact pathways may 
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be potentially large 7 , so that uncertainty figures given in the 
following likely represent lower bounds. 

Along with the number of assumptions and pathways, the 
uncertainty in impact or externality assessments increases 
for subsequent modeling stages towards the endpoint. 8 Take 
for example the 'problem areas' of the ExternE study: cli¬ 
mate change and radioactive exposure (see Fig. 1): While 
emissions of greenhouse gases are relatively well known, the 
estimates of climate change and associated impacts vary 
much more. Typical uncertainties of emissions figures are 
about 10% for energy-related C0 2 , but higher for other 
greenhouse gases and other sources, such as 50%-100% 
for CH 4 from agriculture or CO z from land use changes [26]. 9 
On the far end of the pathway, estimates of economic dam¬ 
ages from climate change commonly vary at least by a factor 
of two, sometimes by more than one magnitude [29-33]. The 
same holds for the radioactivity pathway: releases (at least 
those during normal plant operation) are better understood 
than exposure, doses, and incidents. A similar spread of un¬ 
certainty can be observed for other impact pathways, for ex¬ 
ample potential doses from toxic emissions [34], or fatalities 
resulting from occupational and accidental health risks [35]. 

Given the numerous sources of uncertainty and variability, 
it is not surprising that studies of comparatively similar en¬ 
ergy technologies come up with significantly different exter¬ 
nal cost values [36-39]. Using multivariate regression, 
Sundqvist [40] shows that the variability in electricity exter¬ 
nality estimates caused by methodological and scope dis¬ 
crepancies is comparable to that due to fuel types as differ¬ 
ent as coal and solar, and that site specificity has a minor 
influence. Even within the same study (ExternE) on a par¬ 
ticular technology, estimates have changed significantly over 
time [16]. As to be expected, diverging or changing num¬ 
bers may lead to contradictions, and generally erode the cred¬ 
ibility of such assessments. Consequently, in the ExternE 
study, improvements regarding uncertainty are amongst the 
main recommendations for future research. 

2 How Uncertainty Affects Endpoint Decision-making: 

Statistical Hypothesis Testing 

In the ExternE study, uncertainty is expressed as the stand¬ 
ard deviations Ae (or the 67%-confidence interval) of 


7 Shlyakter [22] offers a quantitative analysis of unsuspected errors re¬ 
vealed by increased knowledge overtime: Examining empirical time se¬ 
ries of measurements, this author concludes that uncertainties are often 
underestimated, and that exponential distribution functions are able to 
incorporate potential future unsuspected errors, based on past overcon¬ 
fidence (see also [23]). 

8 This principle is well-recognised in the literature: Udo de Haes ([9] p. 4) 
state that “in general, definition of an indicator closer to the environmental 
interventions will result in more certain modelling, but will render the indica¬ 
tor less environmentally relevant". Rosa et al. ([24] p. 1506) note that “there 
is a trade-off among indicators. On the one hand, the indicator should be as 
close as possible to the actual impacts [of climate change], i.e. damages. 
On the other hand, it should be calculated with certainty and therefore at 
the beginning of the cause-effect chain". Jolliet et al. [25] paraphrase this 
situation as “a dilemma between certainty and completeness". 

9 Uncertainties in greenhouse gas inventories, and especially in trends, 

affect verification of projects under the Clean Development Mechanism 
and Joint Implementation, as well as compliance with the Kyoto Proto¬ 
col, and emissions trading [27,28]. 


lognormally distributed damage cost e around their geometric 
mean e ([41]. Presumably, unknown systematic errors due 
to the truncations in depth and breadth, or due to lack of 
knowledge are not considered. 10 The standard deviations 
are estimated (by error propagation) to be in excess of ±1 
order of magnitude ([43] p. 310; [44] pp. xviii-xxi). 

Assuming two energy supply alternatives with two impacts 
pathways proceeding along mutually exclusive pathways, 
this estimate implies that their damage cost have to differ by 
at least one order of magnitude in order to be distinguish¬ 
able statistically. More precise: Assuming a confidence in¬ 
terval of 67% = 2 / 3 , then only if nominal damage cost differ 
by more than one order of magnitude, the probability of 
mistakenly distinguishing the two alternatives while in real¬ 
ity they perform equally is less than (1-67%) = V 3 . In statis¬ 
tics parlance, this is expressed as a so-called 'type-I error 1 : 
the probability that the null hypothesis (H 0 : <?j = e 2 ) is re¬ 
jected but true. In scientific hypothesis testing, a 95%-or- 
higher confidence level is usually chosen, with correspond¬ 
ing type-I error probabilities of 5% and lower required. 
Under these conditions, the damage cost of two alternatives 
have to differ by at least 1.96 = 2 orders of magnitude in 
order to be distinguishable. 11 

Consider the following example: In the ExternE assessment of 
nuclear and wind power, the only common impact pathways 
are accidents. These impact pathways are mutually exclusive, 
since the accident causes in the two cases are completely unre¬ 
lated. Externality cost values given are e nucl = 0.1 ECU/GWh, 
and e wind = 0.22 ECU/GWh. 12 Because of errors being log- 
normally distributed, one can subject the hypothesis of 
| l°g(Cuci) _ l°g(®wind) | being null to a t-test, by comparing 


10 Due to limitations in depth and breadth, ExternE estimates have been 
described as 'sub-totals' ([42] p. 181). 

"This argument needs qualification: Rabl and Spadaro ([41] p. 44) state 
that “it is instructive to distinguish between policy decisions that are bi¬ 
nary (e.g., the choice between nuclear or coal-fired power plant) or con¬ 
tinuous (e.g., what limit to set for the S0 2 emissions from a power plant)." 
They argue that "for binary decisions, the situation is sometimes simple 
because the uncertainty, even if large, has no effect if it doesn’t change 
the ranking.' 1 In my view, this argument only holds if the same impact 
pathway is appraised for both alternatives: For example, if a gas-fired 
plant emits less greenhouse gases than a coal-fired plant, then its im¬ 
pact on climate change will be lower, no matter the uncertainties for 
determining climate change impacts. However, when comparing the ra¬ 
diological impact pathway of a nuclear plant with the climate change 
impact pathway of a wind plant, uncertainties are completely uncorrelated, 
so that - purely stochastically - the true radiological impact value may 
be below the mean, while the true climate impact value might be above 
the mean; hence, any ranking will be affected by uncertainty. In the same 
paragraph, Rabl and Spadaro [41] also say that "furthermore, the exter¬ 
nal cost of nuclear are so much lower than those of coal that the ranking 
is not affected by the uncertainties". This corresponds to the situation 
described above of two alternatives scoring so differently that they are 
distinguishable, even under large uncertainty. A similar but only qualita¬ 
tive argument was made by Hertwich et al. ([45] p. 270). The concept of 
overlapping distributions is also exploited in the theory of fuzzy sets [46, 
47], and the overlap of the distributions in Fig. 1 corresponds to the 
intersection of fuzzy sets. 

12 ECU = European Currency Unit, GWh = Gigawatt-hour = 10 9 Watt-hours. 
External cost for the wind fuel cycle were calculated from average occu¬ 
pational accident cost (0.13 ECU/GWh) and public accident cost (0.09 
ECU/GWh). External cost for the nuclear cycle are the consequence of 
radiological doses. 
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£ = |l°g( e nucl ) - Io g(%ind )|/V A 0°g e nucl f + A ( lo gf t0 ^ Ul- 

verse of the t-distribution. Inserting A(log e nucl ) = A(log e wiJld ) = 1, 
e = 0.24. Assuming that the means ^ lucl and were deter¬ 
mined from a large sample, and applying the f-statistic for arbi¬ 
trarily large degrees of freedom, t d2 = e for a = 0.81. Hence the 
type-I error probability of mistakenly favouring one alternative 
when in reality they are indistinguishable is as high as 81%. 
This example may explain why one ExternE report states that 
"the best estimate of any damages value is therefore, on its own, 
inadequate for most policy making purposes" ([48] p. 87). 

This situation of uncertain and therefore indistinguishable 
endpoint measures is likely to arise, especially in the case of 
complex environmental and social impacts. Decision-mak¬ 
ing may only be possible when alternatives exhibit consider¬ 
ably different scores, but in these cases the decision is likely 
to be so obvious that a sophisticated analysis would seem 
out of place. For example, Krewitt ([16] p. 842) states that 
"in spite of the existing uncertainties ExternE results were 
quite robust with regard to the relative ranking of different 
electricity generation technologies". However, this finding 
refers to a comparison of sources as different as coal, lig¬ 
nite, gas, nuclear and wind, with lignite and coal perform¬ 
ing up to one order of magnitude worse than the remainder. 
Lignite and coal show a ranking reversal, and gas and nu¬ 
clear are probably statistically indistinguishable. In this case 
it could be argued that one does not require the amount of 
research expended for ExternE to conclude that wind causes 
lower impacts than coal. 

3 Midpoint Decision-making and Uncertainty: 

Applying Multi-criteria Methods 

Facing prohibitively high endpoint uncertainty, one could re¬ 
vert to a midpoint level where indicators are sufficiently cer¬ 
tain 13 , and use multi-criteria decision (MCD) methods that 
make subjective trade-offs between midpoint categories ex- 


13 The debate around the midpoint/endpoint nexus has been especially 
prominent in the LCA literature [49]: While the SETAC [50] and the UNEP 
Life Cycle Initiative [25] approaches stop quantitative modeling relatively 
early in the cause-effect chain in order to limit uncertainties, the Eco- 
Indicator [3] and the Japanese National LCA project [51] try to enhance 
the relevance of the results by indicating damages through endpoint weight¬ 
ing. Comparing mid- and endpoint approaches is also the subject of the 
ReCiPe project funded by the Dutch government [52], While the ReCiPe 
project advocates the use of sensitivity analysis, and certain scenarios, 
archetypal attitudes or 'world views', it explicitly does not deal with error 
propagation and statistical analysis for decision-making. 


plicit 14 . In Appendix A (online edition only, see http://dx.doi. 
org/10.1065/lca2005.04.201 ). an uncertainty calculus is de¬ 
veloped for applying a MCD method to midpoint levels of 
ExternE impact pathways for two electricity generation alter¬ 
natives. In this case the Analytic Hierarchy Process (AHP) is 
used 15 , but the procedure is equally applicable for other MCD 
techniques. In the following, the AHP will be applied to an 
example decision derived from the ExternE study. 

3.1 A simplified example 

In the following, two electricity supply alternatives investi¬ 
gated in the ExternE study will be subjected to the AHP: 
nuclear and wind energy. For both alternatives, only two 
priority pathways will be considered: a) cancers from (mainly 
global, long-term, low-level collective C-14) radiological 
doses, and b) climate change impacts from fuel-combustion- 
related C0 2 . Note that the examples given below represent 
an extremely simplified picture of the outcome of the ExternE 
study, for example in the way that the pathways were as¬ 
sumed to be linear, with constant conversion factors f t . While 
the magnitude of the impact values e corresponds to the 
ExternE values, some of the midpoint impacts and uncer¬ 
tainties are based on other studies. 16 

The nuclear priority pathway (Table 2, compare Fig. 1) starts 
with releases of radioactivity during normal operation and 
as a consequence of accidents. Multiplication by conversion 
factors /] leads to mid- and endpoints, and associated im¬ 
pacts e (ranges given in brackets). The uncertainties are ex¬ 
pressed in logarithmic terms, that is A(log e) = 1 denotes an 
uncertainty of one order of magnitude. 


14 For a discussion of the role of MCD tools within LCA see [53-55]. Noh et 
al. [56] compare three MCD methods applied in LCA. MCD tools were 
also suggested in an ExternE newsletter [57] for dealing with non-envi- 
ronmental externalities such as from nuclear proliferation risks. 

15 The AHP has been criticised for shortcomings such as rank reversal, its 
lack of reference points, and the independence of weights from attribute 
ranges [58,59]. Since these are problems that are unrelated to the topic 
of this article, they shall not be dealt with here. 

16 The author is also aware of and agrees with Rabl et al.'s ([41] p. 44) 
warning that providing definite values for uncertainties implies "a false 
sense of precision, unjustified in view of the need to use subjective judg¬ 
ment to compensate the lack of information about sources of uncertain¬ 
ties and probability distributions". Furthermore, it may not always be pos¬ 
sible to find a range for impact pathway parameters, let alone a probability 
distribution. Therefore, uncertainty may be better conveyed by descrip¬ 
tive labels such as 'high', 'medium' and 'low' confidence, and/or by ap¬ 
plying fuzzy set methods [47,60,61]. 


Table 2: Simplified impact pathway for radiological impacts of nuclear power (after [43]). All values are normalised to 1 TWh of electrical output 


Pathway stage / Conversion 

Impact e 

Range 

Unit 

A(log e) 

Radioactivity release / f 3 = 0.01 GBq/GBq 

91.5 

(61-137) 

GBq/TWh 

0.18 

Exposure / f 5 = 14.2 man.Sv/GBq 

0.92 

(0.4—2.3) 

GBq/TWh 

0.40 

Dose / f 7 = 0.17 inc./man.Sv 

13.0 

(2.3-72.7) 

man. Sv / TWh 

0.75 

Cancer / k = 158 ECUdisc/inc. 

2.21 

(0.3-14.5) 

inc. / TWh 

0.82 

Externality cost 

350 

(48-2,544) 

ECUdiso / TWh 

0.86 


Notes: GBq = Gigabequerel = 10 9 decays/second; TWh = Terawatt-hour = 10 12 Watt-hours; man.Sv = man-Sievert is a unit for the average individual 
radiological dose equivalents; inc. = incidents; ECU = European Currency Unit is the monetary unit used in the ExternE study; disc = discounted. The 
A(log e) were calculated from geometric standard deviations given by [41]) 
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Table 3: Simplified impact pathway for climate change impacts of wind power (partly after [44], and [24]). All values are normalised to 1 TWh of electrical output 


Pathway stage / Conversion 

Impact e 

Range 

Unit 

A(log e) 

GHG emissions / U= 0.48 10 -9 ppm/T CO 2 

9.1 

(8-11) 

t C0 2 / TWh 

0.08 

GHG concentration / fe= 3.481 C yr/ppm 

4.4 

(3-6) 

10~ 9 ppm/TWh 

0.13 

Global warming / f 8 = 49.17 ECU/t C yr 

153 

(62-374) 

t C yr /TWh 

0.39 

Climate change / /) 0 = 0.2 ECU d iso/ECU 

7,500 

(1,382-40,689) 

ECU /TWh 

0.73 

Externality cost 

1,500 

(240-9,375) 

ECUdisc/TWh 

0.80 


Notes: GHG = greenhouse gas; t = tonnes; TWh = Terawatt-hour = 10 12 Watt-hours; ppm = parts per million; t C yr = tonne-years of carbon is a 
measure of cumulative global warming ([24]); ECU = European Currency Unit is the monetary unit used in the ExternE study; disc = discounted 


For the nuclear impact pathway it is assumed here that hu¬ 
mans are exposed to f 3 = 1% of the released radioactivity. 
For inhalation and ingestion of C-14, the dose conversion is 
f 3 x f s = 0.142 man.Sv/GBq, integrated over 100,000 years 
([43] p. 50). Determining f 3 involves Gaussian plume, tra¬ 
jectory, and deposition modeling. In the simplified dose-re¬ 
sponse relationship f 7 , fatal and non-fatal cancers were 
counted, but not hereditary effects. Systematic errors, for 
example due to choosing the wrong type of relationship, are 
not accounted for ([41] p. 41). The final externality cost of 
350 ECU disc /TWh are the logarithmic mean between the 
extremes of 50 and 2500 ECU disc /TWh calculated for 0% 
and 10% discount rates ([43] p. 200). Midpoint uncertain¬ 
ties were taken from [41] Table 3. 17 Note that considering 
the near-impossibility of estimating parameters and impacts 
for thousands of years into the future, the above enumera¬ 
tion may well be meaningless [16]. 

The wind alternative's priority impact pathway (Table 3) 
deals with greenhouse gases emitted during the plant opera¬ 
tion as well as embodied in the turbine infrastructure. Sys¬ 
tematic errors due to the truncation of the system boundary 
[64] are not taken into account. The climate change impact 
pathway can be expressed in many different ways; however, 
the way given here suffices to provide an example for multi¬ 
criteria decision-making under uncertainty. The increase of 
atmospheric greenhouse gas concentrations is simplified as 
a multiple of cumulative emissions (compare [36] p. 426). 
Global warming impacts are expressed as temporally inte¬ 
grated atmospheric concentrations ([24] p. 10). Climate change 
impacts are manifold, involving a range of midpoints with 
varying units, of which undiscounted cost were chosen here. 
Discounting over several centuries (as is necessary for cli¬ 
mate change impacts) typically involves a cost reduction of 
a factor 5-10, leading to the final estimate from the ExternE 
study ([43] p. 118). Emissions are assumed to be known with 
10-20% uncertainty 18 ([26]). Concentration and global 


17 lt has been recognised that the externality cost calculated in the above 
fashion do not reflect society's willingness to pay to avoid a nuclear ac¬ 
cident, and that not only expert opinion and engineering calculus (as in 
[35]), but also the public's aversion of (voluntarily incurred or imposed) 
risk should influence the probability distributions [48,62], For example, 
damage measures perceived by people at risk are likely to exceed those 
expected by engineers based on probability and likely cost [10,63], and 
large-impact low-probability risks are less tolerated because the magni¬ 
tude of the potential damage exceeds funds or capacities ([37] p. 522). 
This type of externality is not implemented in ExternE, and will therefore 
not be considered at this stage, but in the discussion. 


warming uncertainty were taken from trajectories given in 
the third assessment report of the Intergovernmental Panel 
on Climate Change (IPCC, [65]). The geometric standard 
deviation of external costs is cr g = 2.5 ([41] p. 42), so that 
A(log e ) = log = 0.80 . The uncertainty of climate change 

impacts was derived by subtracting valuation- and discount¬ 
ing-related uncertainties (given in [41] Table 3) from external 
cost uncertainties. It is important to keep in mind that ExternE 
as well as IPCC sources state that ranges of damage cost are 
not based on a comprehensive scientific uncertainty appraisal 
and at best incomplete, and that strictly speaking, no central 
figure can be endorsed as a most likely estimate ([48] p. 63). 

3.2 Applying the AHP 

In order to illustrate uncertainty trade-offs, a relatively cer¬ 
tain indicator ('jobs') is added to the more uncertain indica¬ 
tors 'cancer' and 'climate'; let these three indicators repre¬ 
sent societal goals. As a first step, a decision-maker may be 
asked to specify a required maximum uncertainty level, say 
a maximum type-I-error probability of 0.33. Second, based 
on pairwise comparisons of preferences, AHP weights are 
elicited. 19 Eqs. 1 to 6 in Appendix A (online edition only, see 
http://dx.doi.org/10.1065/lca2005.04.201.1 ) can then be 
applied to performances P, ; of the alternatives j in terms of 
indicators i, in order to obtain normalised performances p tj 
= Pjj I £ pPt , scores s ; = T i w ; p l; , and uncertainties A(ln s ; ). 

The result of the AHP process shows that when evaluated at 
endpoints, wind power scores slightly higher (that is better) 
than nuclear power (Table 4). However, the /-test (rightmost 
columns) shows that the type-I error probability of the overall 
score is so high (0.53) that the decision-maker's requirement 
is not met: a decision under these circumstances is virtually 
impossible. Moreover, quoting an endpoint result in this case 
may only distract from the existing prohibitive uncertainty 
(compare [70] p. 214). Note that the employment indicator 
alone allows a clear distinction, however when determining 
the aggregate score this certainty is swamped by the more 
uncertain indicators. 


is A(log e) = log - log e = log = log —— = log 1.2 = 0.08 

e e 

19 Note that in this case, the common AHP practice of eliciting weights 
independently of attribute ranges is applied. However, it has been ar¬ 
gued that under certain conditions judgments are actually range-sensi¬ 
tive [58,66-69], 
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Table 4: Decision-making near endpoints, at midpoints, and at intervention levels. Uncertainties are expressed as orders of magnitude. Units for green¬ 


house gas and radioactivity pathways as in Table 2 and 3, 'jobs' in arbitrary units 


Level 

Indicator 

Perfo 

Nuclear 

rmance 

Wind 

Relative pc 

Nuclear 

‘rformance 

Wind 

x AHP 

weights = 

Sc 

Nuclear 

ore 

Wind 

t 

type-I 

error prob 

c 

Jobs 

100 ±0.04 

60 ± 0.04 

0.63 ± 0.02 

0.38 ± 0.04 

0.2 

0.13 ±0.02 

0.08 ± 0.04 

5.20 

0.00 

Q_ 

■o 

Cancer 

2.21 ±0.82 

0.22± 0.82 

0.09 ± 1.05 

0.91 ±0.11 

0.5 

0.05 ± 1.05 

0.45 ±0.11 

0.95 

0.34 

0 

Climate change 

750 ± 0.73 

7500 ± 0.73 

0.91 ±0.09 

0.09 ± 0.94 

0.3 

0.27 ±0.09 

0.03 ± 0.94 

1.05 

0.29 

0 

C 






Totals 

0.44 ±0.12 

0.56 ±0.10 

0.63 

0.53 


Jobs 

100 ±0.04 

60 ± 0.04 

0.63 ± 0.02 

0.38 ± 0.04 

0.2 

0.13 ±0.02 

0.08 ± 0.04 

5.20 

0.00 

c 

o 

Dose 

13.0 ±0.75 

1.3 ±0.75 

0.09 ±0.96 

0.91 ±0.10 

0.5 

0.05 ± 0.96 

0.45 ±0.10 

1.04 

0.30 

T3 

Global warming 

15.3 ±0.39 

152.5 ±0.39 

0.91 ± 0.05 

0.09 ± 0.50 

0.3 

0.27 ± 0.05 

0.03 ± 0.50 

1.99 

0.05 







Totals 

0.44 ±0.10 

0.56 ± 0.08 

0.75 

0.45 

C 

Jobs 

100 ±0.04 

60 ± 0.04 

0.63 ±0.02 

0.38 ± 0.04 

0.2 

0.13 ±0.02 

0.08 ± 0.04 

5.20 

0.00 

c 

0 

Exposure 

0.92 ± 0.40 

0.09 ± 0.40 

0.09 ±0.51 

0.91 ±0.05 

0.5 

0.05 ± 0.51 

0.45 ± 0.05 

1.94 

0.05 

0 

GHG concentration 

0.44 ±0.13 

4.38 ±0.13 

0.91 ± 0.02 

0.09 ±0.16 

0.3 

0.27 ± 0.02 

0.03 ±0.16 

6.06 

0.00 

C 






Totals 

0.44 ± 0.05 

0.56 ± 0.04 

1.44 

0.15 


In order to reduce the type-I error probability to below 0.33, 
the decision could be located closer to the interventions lev¬ 
els, for example using the indicators 'dose' and 'warming 1 , 
or 'exposure' and 'concentration' instead of 'cancer' and 'cli¬ 
mate'. Note that relative performances and scores are now 
associated with lower uncertainties. As a result of the simplified 
linearisation of the impact pathway (see Table 2 and 3), wind 
power still scores better, but in practice this need not be the 
case. The main point is that a choice between nuclear and 
wind is more certain at only 0.45 and 0.15 type-I error prob¬ 
abilities, respectively. Basically, the decision-maker has given 
up the objectives 'cancer' and 'climate', because of a clear 
lack of knowledge and understanding, even though these 
objectives might have matched more closely the societal 
goals. The new objectives are to reduce exposures and con¬ 
centrations, and their only remaining link to the societal goals 
is that they are known to be positively related. The deci¬ 
sion-maker's shift to this new set of objectives makes per¬ 
fect sense, since the uncertainty of the old objectives pre¬ 
cluded any decision. 

3.3 Relevance 

In spite of their increased certainty, it has been argued that 
indicators at the intervention or midpoint level are only 
'proxy attributes' ([71] p. 112), and carry 'a hidden uncer¬ 
tainty in their [environmental] relevance' ([9] p. 8), and that 
choosing indicators closer to interventions levels only moves 
the uncertainty from the numbers to the (environmental) 
relevance. This comes about because important factors in 
the cause-effect chain of the impact pathway might be left 
out of the midpoint information, or because weights may be 
more uncertain at midpoints than at endpoints. 

In order to allow the decision-maker to address the more tan¬ 
gible and relevant endpoint question, even though affected by 
systematic epistemological uncertainty, Keeney et al. ([72]; as 
cited in [71]) suggest replacing midpoint impacts p tj with deci¬ 
sion-makers’ midpoint utility M-(p-) = J w ; .(D ; .) prob(D-lp-) dp ;/ , 
where D. are endpoint damages of midpoint impacts p :j , 


« ; -(D ; -) the decision-maker's utility function for endpoint 
damages, and prob(D (/ lp /; ) is the conditional probability of 
D, ; occurring as as result of p if . Scoring then proceeds as 
usual via s ; = Z,- w { u tl (p t) ). Acknowledging that a comprehen¬ 
sive enumeration of endpoint damages is impossible, the idea 
is basically to use an elicitable utility function ujp-) in place 
of the unknown final part of the impact pathway D i; = /(p ;; ). 
However, this formulation still faces the difficulty of having 
to enumerate probjD-lp-), and relevant information is once 
again highly uncertain. 20 

The certainty-relevance trade-off seems in fact pivotal in the 
midpoint/endpoint debate in LCA [45,49]: In order to an¬ 
swer one endpoint question, uncertain endpoint damages 
may tell as little as more certain but less relevant midpoint 
numbers. The following Section discusses further advantages 
and disadvantages of mid- and endpoints and, notwithstand¬ 
ing the seemingly inevitable certainty/relevance trade-off, 
gives some arguments in favour of multi-criteria decision¬ 
making at midpoints. 

4 Discussion 

4.1 Midpoints or endpoints? 

With respect to whether mid- or endpoint indicators are fa¬ 
vourable, Heijungs et al. [52] elaborate: "Many claim that 
endpoint indicators are easier to understand and thus inter¬ 
pret, as they express issues of societal concern, such as hu¬ 
man health damage or ecosystem quality damage. This means 
that the uncertainty in the interpretation can be potentially 
lower compared to a midpoint method, especially in deci¬ 
sion-making. Others claim, however, that midpoint indica¬ 
tors are easier to understand and thus interpret, as they re¬ 
late to facts and phenomena, such as decrease of the ozone 
layer, whereas endpoint indicators pertain to badly-defined 
and subjective intuitive notions, such as human health". 


“Considering the complexity of the midpoint-endpoint chain, Rahimi et al. 
[61 ] use linguistic fuzzy variables that allow for vagueness about endpoint 
damages. 
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In my view, decision-making at midpoints has several ad¬ 
vantages: 21 First, instead of providing a few aggregated num¬ 
bers, the more multi-facetted midpoint information clearly 
reveals the multi-dimensionality of the problem at hand, and 
the trade-offs between the inherent aspects. Appropriately 
chosen MCD methods can aid the decision-maker if the amount 
of information exceeds cognitive limits. Second, compared to 
midpoints, endpoint assessments require additional steps of 
data collection, modeling and computation, and hence require 
more time, labour and resources, with potentially little gain in 
decision certainty. Third, as highlighted by the example of the 
employment indicator in Table 4, aggregation of impact cat¬ 
egories and pathways may cause uncertain components to 
swamp certain components. In contrast, reverting to midpoint 
levels opens the opportunity of carrying out an iterative pro¬ 
cedure, where too uncertain indicators are simply excluded. 
Consider again Table 4 and assume that the decision-maker 
insists on near-endpoint indicators: Given that with regard to 
'cancer' and 'climate change', the two alternatives are indis¬ 
tinguishable within the certainty required, a decision could 
sensibly be based on 'jobs' alone, because in terms of this cri¬ 
terion at least a difference can be discerned. 

Fourth, and maybe most importantly, multi-criteria decision 
analysis at midpoint levels is able to include characteristics 
that impact modeling and valuation has trouble quantify¬ 
ing. The decision-making literature offers a rich repertoire 
of factors that significantly influence people's decisions, but 
are difficult or impossible to quantify. Stirling ([37] p. 518) 
provides an extensive list of problems faced by externality 
assessments that "might be summed up as 'a failure to ad¬ 
dress the multidimensional nature of environmental ap¬ 
praisal'". 22 In addition to the uncertainty aspects that are 
the topic of this article, Stirling and others [45,75-77] find 
that aggregate external cost estimates lack representation of 

- the spatial, social and temporal distribution of impacts, 
burdens and benefits (international, inter-generational 
and inter-societal equity); 

- the autonomy of the people affected (voluntariness, con¬ 
trollability and familiarity of exposure); 23 

- perceptions of severity, frequency, immediacy and revers¬ 
ibility of potential impacts; 24 

- perceived risk (as opposed to calculated risk based on en¬ 
gineers' expectations of magnitudes and probabilities); 25 


21 Without going into numerical and methodological details, Udo de Haes 
[73] comes to some of the above conclusions when recommending so¬ 
cial panels evaluating midpoints as the preferred approach for weighting 
in LCA. He also qualifies the finding by pointing out that the decision 
may be affected by who is a panel member, and where and when the 
decision is made. 

22 See also a paper by Watson [74] on the different origins of monetisation 
(as in cost-benefit analysis) and decision-analytical tools in economics 
and psychology, respectively. 

23 Examples: voluntariness - worker versus general public, controllability 
- risks from car travel versus risks from living near a power plant, famili¬ 
arity - expert's versus lay person's knowledge. 

24 Examples: severity and frequency - reactor accident versus car acci¬ 
dents, immediacy - air pollution and immediate respiratory diseases ver¬ 
sus delayed climate change, reversibility - noise versus species loss. 

25 Freeman [78] makes the point that it is common and accepted to base 
investment decisions under uncertainty on ex-ante criteria such as risk 
aversion. 


- ethical, moral, religious and political beliefs and princi¬ 
ples; 

- perceived incompleteness of human knowledge. 

For example, facing a choice between nuclear and wind 
power where damages are indistinguishable and impact path¬ 
ways complex, decision makers may actually consider those 
aspects they know and that distinguish the alternatives. For 
example, in Australia, radioactivity evokes images of con¬ 
taminated rivers in Jabiluka (a Uranium mine in Kakadu 
National Park) and sick animals and people, while green¬ 
house gas emissions spell droughts, storms and bleaching 
corals. None of these associations are true endpoint meas¬ 
ures. One could argue that when ranking the relative impor¬ 
tance of nuclear versus wind power, respondents may actu¬ 
ally think of prominent but proxy midpoint impacts when 
passing their judgment [79], and not complete impact path¬ 
ways or damage costs. They will likely judge the alterna¬ 
tives with respect to voluntariness, severity, immediacy etc. 
For example, people may prefer the relatively certain pros¬ 
pect of struggling in a warmer world to the small possibility 
of being struck by a radiological cancer leading to a grue¬ 
some death. Or, they may not trust the authorities adminis¬ 
tering nuclear power in guaranteeing safe operation or pre¬ 
vention of proliferation. Or, they may think of dumping 
radioactive waste on future generations for unimaginable 
times to come violates an ethical principle that has no trade¬ 
offs. These and many other thoughts may actually be quite 
clearly formed and certain, leading respondents to weight 
'radioactivity' as more important than 'greenhouse gas emis¬ 
sions'. Note that in order to judge many of the above as¬ 
pects, people do not need extensive knowledge about im¬ 
pact pathways and damage cost. 

4.2 How uncertainty affects the choice of MCD method: 

Range-sensitivity of weights 

Uncertainty bears not only on choices between mid- and 
endpoints, but also on choices between MCD methods. 
Some authors maintain that the elicitation of weights using 
questions that purely relate to attributes without any refer¬ 
ence to ranges (such as in the AHP and Table 4) are mean¬ 
ingless, and that preferences depend on the available ranges 
[58,68]. For example, in order to answer the question 'What 
is more important in a car: cost or colour?', one would 
want to know how expensive and colourful the cars are 
that can be chosen from. One would expect that for many 
people cost is more important in a purchase decision. How¬ 
ever, if all cars cost about the same (small range), but had 
vastly different colours (large range), then a choice would 
be made on colour. 

However, for attributes such as climate change or cancer, 
where endpoint ranges are extremely broad, and midpoint 
ranges (emissions and releases) are difficult to interpret, re¬ 
spondents may actually weight attributes independent of 
ranges, that is they switch from preferences within the nar¬ 
row range and context of alternatives to what Fischer [69] 
calls generalised notions about attributes and range-inde- 
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pendent, 'global' preferences, with a maximum spectrum of 
situations in mind. 26 

Range-sensitivity or insensitivity of weights bears on the 
choice of multi-criteria decision method: Interpreting the 
weighted sum s ; = S, w t p tj as a utility function imposes a 
specific utility trade-off interpretation on the weights 
placing them in the context of the attribute ranges of alter¬ 
natives [66,67,69]. This means that, in utility-based methods, 
weights cannot be elicited independently from attribute 
ranges. In contrast, AHP performances p tj = P^‘ / Z, p£‘ can¬ 
not be interpreted as cardinal utilities, and the weights are 
chosen independently from performance ranges ([67] p. 574). 

Considering that the AHP is said to be particularly suited to 
deal with intangibles that do not lend themselves for easy 
translation into utility terms [81], and that are vague and 
uncertain, global preferences are likely to occur. These glo¬ 
bal preferences (or Fischer's generalised notions) may refer 
to the beliefs, principles, rights and feelings listed in the pre¬ 
vious section, and may therefore be 1) independent of ranges, 
and 2) independent of mid- or endpoints. Given 1), the AHP's 
unreferenced ratio-scale approach may not be so inappro¬ 
priate as critics of the AHP's lack of meaningful reference 
intervals [58,59,68,82] purport. Given 2), midpoint weights 
and endpoint weights based on global preferences may ac¬ 
tually be identical, as in Table 3 (compare [79] p. 210). 

5 Conclusions 

Applying statistical hypothesis testing to the example of 
impact pathways in the ExternE study on external cost of 
electricity supply alternatives, this article has quantitatively 
shown that impact and external cost assessments featuring a 
single, aggregated endpoint measure can be so uncertain that 
neither cardinal differences between, nor ordinal rankings of 
alternatives retain sufficient significance while moving along 
the impact pathway. As a result, as Eyre [83] states, "it seems 
an inescapable conclusion that, for some important impacts, 
reliable monetary valuations are not a realistic objective". For 
example, even though expected damages from climate change 
often dominate impact and cost rankings, they may only be 
'considered as a helpful indicator of environmental problems 
rather than an input into cost-benefit analysis' [42]. 

Moreover, presented as discrete values, figures for endpoint 
damage cost, and differences between alternatives can be 
deceptive if seen without a clear understanding of the uncer¬ 
tainties involved. In this respect, ExternE authors state that 
"there is a need for other users of the results [...] to both 
understand and pass on information regarding uncertainty, 
and not to ignore it because of the problems that inevitably 
arise. The Project team feels so strongly about this that ref¬ 
erence should not be made to the ExternE Project results 
unless reference is also made to the uncertainties inherent in 
any analysis and our attempts to address them" ([48] p. 63). 


26 Fischer et al. [79] observe a bias in respondents weighting proxy attributes 
rather than endpoints. This bias will in general depend on whether the 
endpoint range expected from the proxies lies within the lower or upper 
part of the respective globally perceived (for example maximum possible) 
endpoint range. Hammitt [80] shows that climate change abatement policy 
outcomes may be affected by whether mid- or endpoints are evaluated. 


Notwithstanding it falling short of its ambitious objective, 
the endpoint modeling in the ExternE project has demon¬ 
strated that external cost of some electricity generation al¬ 
ternatives can be potentially large. The associated high uncer¬ 
tainties can be a valuable result in themselves, because they 
convey a risk that, seen in the context of the precautionary 
principle, can stimulate action towards implementing clearly 
sustainable technologies as a means of hedging against the 
worst scenarios [83]. Finally, endpoint and impact pathway 
knowledge can aid weight elicitation at the midpoint level. 

Still, for the purpose of decision-making, it may not make 
sense to expend resources on shedding light on endpoint 
parts of impact pathways, when uncertainty in further mid¬ 
point factors is already prohibitive. As Krupnick et al. ([36] 
p. 428) put it: "The weakest link in the sequence of func¬ 
tions in a pathway sets the standard for the level of detail in 
the analysis. Consequently there may be little benefit from a 
high degree of resolution at one stage that is only lost in the 
final analysis because of less resolution at another stage." 

If endpoints are too uncertain to allow a decision to be made 
with reasonable confidence, an assessment could be shifted 
to midpoint levels. Here however, it is often pointed out 
that the multi-facetted midpoint information is too com¬ 
plex and poses difficulties for decision-makers. Moreover, 
midpoint indicators are generally further removed from peo¬ 
ple's experience, and less relevant to the question that peo¬ 
ple actually want to solve. There is an uncertainty associ¬ 
ated with this diminished relevance which may well equal 
endpoint uncertainty. 

In this paper it has been argued that if the endpoint question 
is unanswerable (with the certainty required by the deci¬ 
sion-maker), a decision should be made on the basis of 
stakeholders' subjective judgments about the more certain 
midpoint levels. This is because in addition to quantitative 
data, there is a wealth of reasons that affect decisions, which 
are based on beliefs, principles, rights and feelings. Many 
people would have relatively clear and certain preferences 
with respect to these beliefs etc, and would insist that these 
are valid reasons that should play a role in decisions. Since 
it is exactly these aspects that endpoint approaches such as 
impact pathway modeling and monetisation are largely un¬ 
able to deal with, multi-criteria approaches applied to mid¬ 
points appear as the preferred option. A conclusion could 
be summarised as: When bridging the midpoint-endpoint 
gap, the difference between using impact pathway modeling 
and monetisation, or using human judgment is often the same 
as that between going by a wild guess or by gut feeling. 
Depending on how wild that guess is, people may actually 
be happier with gut feeling. 
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Appendix A: Uncertainty in the AHP 

(Online edition only, see http://dx.doi.org/10.1065/lca2005.04.201 ) 


Let the performance of alternative j in terms of indicator i 
expressed in its own physical units be denoted by P i; . The 
AHP’s first step is to make the decision problem independ¬ 
ent of units by expressing the performance of alternatives as 
relative to each other, and normalise: 



k 


The exponent /l, indicates whether indicator i is desirable (/l, 
= 1) or undesirable (/l, = -1), so that a high p value indicates 
a good outcome. Second, the overall score s ; of alternative j 
is then the weighted sum over relative performances 


v - X u '/u 

i 


( 2 ) 


where the weights w i are obtained in the conventional fashion 
through a pairwise comparison matrix, yielding an eigen¬ 
vector w of weights that are normalised to satisfy £, w ; = 1. 

Assume that the indicators i may be located either at the 
interventions level, or at mid-, or endpoints. Rabl and 
Spadaro [41] have shown that for indicators resulting from 
multiplicative processes involving independent factors (such 
as in ExternE impact pathways), lognormally distributed ob¬ 
servations around geometric means are most plausible (see 
also [84]). If the Pp in Eq. 1 are lognormally distributed, then 
the and s ; are also lognormally distributed. Their relative 
uncertainties can be calculated by first converting Eq. 1 to 


In p u = In Pf : - In 




V k 


( 3 ) 


where the observations of the In A and In p :j are normally 
distributed so that 




Inserting 




( 4 ) 



k k 


where 8 ]n is the Kronecker symbol with 8 jn =1 if / = n and 8 jn 
= 0 otherwise, Eq. 4 becomes 



V n 


= Ji 1 - Pij ) 2 A ( ln p j ) 2 + X A),'4 n P m J 
V n *j 

The lognormal uncertainty of the scores s- is then 


A(ln Sj ) = 


5(lns ; .) 


A (ln p nJ ) 


( 6 ) 


W „P„ 
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A (ln aJ 


On the other hand, some indicator data (for example em¬ 
ployment) or some dose-response functions ([41] p. 40) may 
derive from adding together survey or census samples that 
are normally distributed around an arithmetic mean. In this 
case, the relative uncertainty of the relative performances p tj 
can be determined analytically through 


A P V 



A P fil 

in 
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Pij Pa 


( 7 ) 


Inserting 
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Eq. 7 becomes 


lognormal distribution so that their confidence intervals 
coincide. 27 


Ap u . 

1 - /? 

I \^f S jMn 

PiiAP.n " 

\P 

Pv 

'M p * 

P 

m y 






+1 

n*j 




Pi, 


in J 


The uncertainty of the scores s ; is then 



Finally, weights w t may also be uncertain: Decision-makers 
may not be willing and/or able to provide specific enough 
or all of the required information, 28 or potential errors may 
occur when soliciting information from respondents, for 
example when respondents have not fully understood the 
definition or the significance of an indicator because of in¬ 
sufficient background information. A further variability may 
arise if the w t represent societal values, with observations 
across respondents and over time distributed around a mean 
[901- Incorporating parametrical uncertainty of the into 
Eq. 9 is straightforward (see [91,92]), but it is not necessary 
to deal with this issue here because it does not add any new 
aspect to illustrating the midpoint/endpoint problem. 


A complication arises when either within one impact path¬ 
way, or amongst indicators within a score, lognormal as well 
as normal distributions appear: Normal distributions allow 
(sometimes implausible) negative values, and mixed distri¬ 
butions can be of complex form, so that uncertainties can 
may not be evaluated analytically. In these cases, Rabl and 
Spadaro ([41] p. 40) suggest approximating a normal with a 


27 lf m is the mean of the normal distribution, and dl2 its 68%-confidence 
interval, the geometric standard deviation s g of the lognormal distribu¬ 
tion is then a\ = (p + 82)(ju - 8l)~ l . 

28 A class of multi-attribute decision methods therefore feature simplified 
procedures in which only a rank ordering is elicited from respondents 
(for example in SMART [85]; or lexicographic methods [86], and weights 
are inferred from ranks via unit weighting [87], rank sums [88] or rank- 
order centroids [89], 
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